Benchmarking Music Information Retrieval Systems

نویسندگان

  • Josh Reiss
  • Mark Sandler
چکیده

Our goal is to create an accurate and effective benchmarking system for music information retrieval (MIR) systems. This will serve the multiple purposes of inspiring the MIR community to add additional features and increased speed into existing projects, and to measure the performance of their work and incorporate the ideas of other works. To date, there has been no systematic rigorous review of the field, and thus there is little knowledge of when an MIR implementation might fail in a real world setting. Benchmarking MIR systems is currently hindered by the diversity of the systems, by their relatively new and unrefined nature, and by the limited number of accessible systems. Thus most of what will be described here will be introductory and will lay down the framework for future benchmarking and analysis. Particular attention will be paid to the evaluation issues surrounding retrieval of audio in test collections.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

MIR Benchmarking: Lessons Learned from the Multimedia Community

Music Information Retrieval may be perceived as part of the larger Multimedia Information Retrieval research area. However, many researchers in Music Information Retrieval are unaware that the problems they deal with have analogous problems in image and video retrieval. Many issues concerning the creation of testbed digital libraries and effective benchmarking of information retrieval systems a...

متن کامل

Prototyping a Vibrato-Aware Query-By-Humming (QBH) Music Information Retrieval System for Mobile Communication Devices: Case of Chromatic Harmonica

Background and Aim: The current research aims at prototyping query-by-humming music information retrieval systems for smart phones. Methods: This multi-method research follows simulation technique from mixed models of the operations research methodology, and the documentary research method, simultaneously. Two chromatic harmonica albums comprised the research population. To achieve the purpose ...

متن کامل

Beyond Recall and Precision: A Full Framework for MIR System Evaluation

The Cranfield tests are perhaps the most well-known and often cited example of benchmarking of information retrieval systems. However, of the six criteria that Cleverdon identified as pertinent for analysis of information retrieval systems, only two, precision and recall, are typically investigated. We argue that the other criteria are also vitally important for advanced IR systems such as a mu...

متن کامل

MusiClef: Multimodal Music Tagging Task

MusiClef is a multimodal music benchmarking initiative that will be running a MediaEval 2012 Brave New Task on Multimodal Music Tagging. This paper describes the setup of this task, showing how it complements existing benchmarking initiatives and fosters less explored methodological directions in Music Information Retrieval. MusiClef deals with a concrete use case, encourages multimodal approac...

متن کامل

Overview of MusiCLEF 2011

MusiCLEF is a novel benchmarking activity that aims at promoting the development of new methodologies for music access and retrieval on real public music collections. A major focus is given to multimodal retrieval that, in the case of music collections, can be obtained by combining content-based information, automatically extracted from music files, with contextual information, provided by user...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2002